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1.0 INTRODUCTION 


The Southeastern Virginia Urban Plume Study (SEV-UPS), an element of 
the NASA Regional Tropospheric Program, is designed to provide air quality 
experiments for the purpose of evaluating maturing NASA remote sensors 
participating in the state-of-the-art air quality experiment. 

The objectives for the 1979 SEV-UPS field program fell into five (5) 
basic categories : 1 ) correlative measurement for comparison between in 

situ and remote sensor data, 2) demonstration of remote sensor applications 
through participation in urban scale air quality experiments, 3) correla- 
tive data missions involving various situ sensors or systems, 4) photo- 
chemical investigations dealing with aging air parcels, and 5) measurement 
of local ozone concentration levels in the Southeast Virginia Region. 

The field measurement effort, designed to provide the data base for 
the above objectives, involved in situ and remote measurements. The net- 
work included airborne platforms, ground stations (air quality), tethered 
balloon sites, acoustic radar and rawinsondes. The monitoring network for 
in situ meteorological and air quality measurements included three (3) air- 
borne platforms, twelve (12) ground stations, two (2) tethered balloon 
sites and two (2) rawinsonde launch sites. 

The purpose of this current study is to provide an assessment of the 
quality of the hi situ data from the 1979 Southeastern Virginia Urban Plume 
Study. The tasks performed as a part of this effort include: 

• Overview of quality assurance and quality control in the 

SEV-UPS program, 

• Tabulation and analysis of audit results, 

• Tabulation and analysis of comparison data of simul- 

taneous measurements made by collocated systems, 

• Summary of specific conclusions (effect of identifiable 

bias), and 

• Overall assessment of 1979 SEV-UPS in situ data base. 


Technical direction was provided by Dr. Gerald Gregory the Technical 
Contract Monitor for this effort. Many of the provisions for Quality 
Assurance described herein were designed into the SEV-UPS program by 
Dr. Gregory and his associates. 

RTI personnel participating in this study in addition to the named 
authors include the following: Dr. Ty Hartwell and Ms. Susan Setter gren, 

statistical data processing; Ms. Dana Payne, Ms. JoAnn Leepard, and 
Ms. Emily Paynter, text typing, editing, and technical illustration. 
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2.0 SYMBOLS AND ABBREVIATIONS 


CC 

CFR 

CH4 

CO 

EPA 

LAS 

m 

ML 

NAS 

NASA 

NBS 

NCC 

NO 

NOx 

NO2 

ppb 

ppm 

QA 

OC 

RTI 

SAPCB 

SAS 

SEV-UPS 

SO2 

SRM 

THC 

UV 


Communications Center 

Code of Federal Regulations 

Methane 

Carbon Monoxide 

Environmental Protection Agency 
Laser Absorption Spectrometer 
Me t e r s 

Monitor Labs 
Naval Air Station 

National Aeronautics and Space Administration 

National Bureau of Standards 

Naval Communications Center 

Nitric Oxide 

Oxides of Nitrogen 

Nitrogen Dioxide 

Parts per Billion 

Parts per Million 

Duality Assurance 

Quality Control 

Research Triangle Institute 

State Air Pollution Control Board 

Statistical Analysis System 

Southeastern Virginia Urban Plume Study 

Sul fur Di oxide 

Standard Reference Material 

Total Hydrocarbon 

U1 traviolet 
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3.0 QUALITY SYSTEM 


3.1 Quality System Concept and Terminology 

The word quality refers to the degree to which a product or service 
satisfies the need for which it is intended (ref. 1). The immediate 
product of air monitoring is data, and data quality is measured in terms of 
the degree of accuracy, precision, completeness, representativeness, and 
comparability of the monitoring data (ref. 2). The total quality system 
(i.e., the collective system of planned quality control and quality 
assurance activities) is designed to improve data quality through the 
incorporation of measures to increase data accuracy and to prevent data 
loss. Quality control procedures include all activities performed by the 
monitoring operation to insure that the collected data are of sufficient 
quality to meet the requirements of the monitoring program. Examples of QC 
activities include zero/span checks, preventive maintenance and in-house 
review of system operation. Quality assurance activities, on the other 
hand, are those activities performed by personnel independent of the opera- 
tional personnel to measure the success of the quality control effort. 
Typical activities falling under the realm of quality assurance include 
independent systems audits, performance audits, data validation by the end 
user of the data and assessment of system performance by an independent 
agency. 


3.1.1 Quality System Activities . - Many of the activities performed 

as quality assurance and quality control are similar with the only 
difference being in the person performing them. For example, a site 

inspection by the operator is a preventive measure and falls under the 
quality control category. If the inspection is performed by an independent 
auditor, it is a systems audit and, as such, is considered to be a quality 
assurance activity. Because of the commonality between QC and QA activi- 
ties, EPA commonly uses the term quality assurance to encompass both cate- 
gories of activities. 
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II I 


EPA has published an extensive list of quality-related activities per- 
formed under a total quality system (ref. 2). Examples of these activities 
include : 


Document Control 

QA Policy and Objective Definition 

Definition of Organization and Responsibilities 

Planning for Quality Assurance 

Training 

Pretest Preparation 

Preventive Maintenance 

Data Reporting Checks 

Calibration and Zero-Span Checks 

Interlaboratory and Intralaboratory Testing 

Auditing 

Data Validation 

Statistical Analysis of Data 

Quality Reports to Management 

The quality system is inseparable from a routine monitoring effort 
since many of the functions (e.g., calibration, preventive maintenance and 
planning) are essential if a monitoring program is to be successful. 

The activities listed above were incorporated in the SEV-UPS program; 
however, in many instances, the less important activities were not formerly 
documented although they were performed. Also, extensive independent 
system audits were not performed on the stations which were established for 
purposes other than the SEV-UPS program (e.g., stations operated by the 
State of Virginia). As a consequence, the detailed operational and QA 
procedures were not documented by RTI. The scope of this document has been 
limited to the documentation of calibration and audit activities since 
these activities yield quantitative information for estimating the accuracy 
of the reported data. 
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3.1.2 Quality Standards . - Accurate and repeatable procedures are 
required for all quality assurance activities. Ideally, all instrument 
calibrations or checks should reference the instrument to a standard whose 
level is invariant and well-defined by some scientific principle (e.g, 
temperature of the triple point of water). However, for many gaseous 
analyzers, field calibration with a primary standard is not feasible or 
practical. In the absence of a primary standard, stable secondary 

standards which are referenced to a primary standard are used. Appropriate 
primary standards for environmental monitoring are maintained by the 
National Bureau of Standards (NBS), U. S. Department of Commerce. Standard 
Reference Materials (which have been referenced to primary standards) are 
also available from NBS for use by monitoring agencies as the highest level 
standard at their installation. Individual calibration standards (i.e., 
secondary standards) may be referenced to NBS primary standards using 
published procedures. This system of traceability for all instruments to a 
single reference provides uniformity within, and consistency between, 
measurements made by all organizations in all parts of the country. 

3.2 SEV-UPS Quality System 

3.2.1 Quality Control . - As stated earlier, the SEV-UPS monitoring 
network involved airborne measurement platforms as well as a network of 
surface monitoring sites. Although the measurement techniques employed in 
an airborne system are similar to those used in a fixed surface system, the 
harsh operating environment of an airborne system requires more intensive 
quality control measures. For example, under ideal conditions in a ground 
station environment, dynamic multipoint calibrations may be performed at 
two week intervals, with daily zero/span checks performed between calibra- 
tions. The airborne operation, however, is considerably more costly than 
the ground operation, and is complicated by changing environmental condi- 
tions (pressure, vibration, etc.). The calibration protocol for an air- 
borne system is therefore intensified over that used for the ground 
station, in order to detect and correct problems that would result in 
costly data loss. 
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3.2.1. 1 Surface Monitoring Systems: The principal quality control 

procedures performed for the surface monitoring systems included : 

• The extended operation and checkout of all analyzers and 

data recording systems at a central location prior to 
the field measurements for RTI/NASA ground stations 
only. 

• Establishment of traceability for calibration/span gas 

standards prior to the field study. 

• Performance of periodic multipoint calibrations. 

• Performance of periodic (often daily) zero/span checks. 

• Maintenance of (often daily) checklists and logs for flow- 

rates, pressure, temperature, equipment failures, 
analyzer zero/span adjustments, etc. 


The zero/span checks served as indicators of instrument drift. 
Adjustments were made to the zero settings if zero drift exceeded 
j+3 percent of the full scale value. Multipoint calibrations were repeated 
and adjustments were made if the instrument span drift exceeded jj^lS percent 
of the full scale from initial calibration within a 24-hour period. 
A detailed description of calibration, including zero/span, procedures and 
a list of standards is provided in reference 3. 


3. 2. 1.2 Airborne Systems: Quality control procedures for the air- 

craft measurement systems included : 

• Analyzer characterization in altitude chamber to determine 

pressure-related changes in analyzer sensitivity and 
zero air response. 

• Multipoint calibration of gas analyzers, pressure and 

temperature measurement systems, 

• Zero/span checks for integrating nephelometer , total 

temperature sensor, 

• In-flight zero air response checks at varying altitudes, 

• Preflight, in-flight and postflight equipment and proce- 

dures checks (checklists), and 

• Postflight data validation procedures. 
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The gas analyzers used on the NASA Cessna and the RTI Navajo were 
tested for altitude effects in an altitude simulation chamber. Instruments 
used on the NASA aircraft were tested during June 1979, two months prior to 
the SEV-UPS field program. Tests were conducted over the range of pressure 
corresponding to altitudes from ground level to approximately 7500 m for 
ozone analyzers, and from ground level to approximately 3000 m for sulfur 
dioxide and oxides of nitrogen analyzers. The products of these tests were 
correction functions which were applied to the aerometric data to com- 
pensate for altitude effects. Procedures for, and results of, these tests 
are given in reference 4. The RTI analyzers were tested in the same manner 
on several earlier occasions and demonstrated repeatable pressure charac- 
teristics. The sulfur dioxide analyzer was retested at the same time as 
the NASA analyzers described above to reverify its characteristics. 

During the field measurement program, calibrations were conducted for 
the gas analyzers every two to three days. The calibrations generally 

bracketed a day of sampling. The actual calibration procedure and 
standards used are described in reference 3. 

3.2.2 Quality Assurance . - Quality assurance activities performed for 
the 1979 SEV-UPS were designed to provide independent, quantitative checks 
of the precision and accuracy of the field monitoring data. The activities 
were as follows : 

• Independent performance audits of the ambient air analyzers, 

• Comparative sampling, and 

• Data validation. 

3.2.2. 1 Performance Audits: During the period of August 12 through 

August 19, 1979, an independent audit team conducted performance audits of 
four (4) aircraft and ten (10) surface air monitoring systems. The objec- 
tive of the on-site performance audits was to collect information on the 
accuracy of the study’s measurements of ozone, oxides of nitrogen, total 
hydrocarbons, methane, carbon monoxide and sulfur dioxide. In addition, 
comparative audits were conducted on wind speed and wind direction sensors 
for selected sites. 
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All the materials (gaseous transfer standards) and comparative 
sampling equipment used in the audits were traceable, in so far as 
possible, to NBS standards. Methods used for establishing traceability for 
the gaseous audit standards were identical to those used for calibration 
transfer standards. 

The methods, standards and pertinent references are presented for each 
measured parameter as follows: 

Ozone 

Audit Method: UV photometry 

Standard: Referenced Dasibi U.V. photometer 

Reference: "Technical Assistance Document for the 

Calibration of Ambient Ozone 
Monitors" (ref. 5) 

Oxides of Nitrogen 

Audit Method: Gas phase titration 
Standard: Cylinder NO, NBS Traceable 

Reference: "Technical Assistance Document for the 

Chemiluminescence Measurement of Nitrogen 
Dioxide" (ref, 6) 

Sulfur Dioxide 

Audit Method: Gas phase dilution 

Standard: Cylinder SO2, NBS Traceable 

Reference: "Use of the Flame Photometric Detector 

Method for Measurement of Sulfur 
Dioxide in Ambient Air" (ref. 7) 

THC, NMHC, CO 

Audit Method: Gas phase dilution 

Standard: Cylinder CH4, NBS Traceable 

Reference: "Reference Methods for the Determination 

of Hydrocarbons Corrected for 
Methane" (ref. 8) 

The results of these audits are given in Section 4.0. 

3. 2. 2. 2 Comparative Sampling: Flight patterns, designed and reviewed 

by NASA personnel prior to the monitoring program, contained provisions for 
simultaneous measurements by two aircraft at nearly the same location. At 
the completion of the program, data from these flight segments could be 
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analyzed to provide an indication of comparability of measurements from 
different platforms. Data from these segments are identified, and the 
results are tabulated, in Section 5.0. 

3. 2. 2. 3 Data Validation: The validation procedure for each moni- 

toring system generally consisted of a review of plots for each monitored 
variable to determine if the processed data are consistent with in-flight 
observations and with other sources of data. At RTI, tests are incor- 
porated in the software to flag certain data discrepancies which physically 
cannot occur (e.g., temperature less than dew point, etc.). The exact data 
validation procedure used varies between monitoring agencies. Detailed 
validation criteria for each agency are not presented here. 
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4.0 AUDIT RESULTS 


During the initial week of the 1979 SEV-UPS study, an RTI audit team, 
independent of the monitoring effort, visited ten ground stations and four 
airborne platforms for the purpose of auditing the situ gas analyzers. 
All ozone, oxides of nitrogen, total hydrocarbon, methane, carbon monoxide 
and sulfur dioxide analyzers were audited at thpse sites. Table 4-1 sum- 
marizes the aerometric variables which were monitored at each of the sur- 
face stations and illustrates which were audited. 

All ground stations were scheduled to be in operation from August 13 
until August 31, 1979, The audits were conducted during the first week of 
this period to insure that: 1) audit data would be representative of 

instrument operation during the program, and 2) audits would be performed 
sufficiently early in the program so that instruments which did not meet 
the required accuracy tolerance could be repaired or recalibrated with 
minimum data loss. 

The RTI personnel performing the audit were not associated with either 
the surface or the aircraft air monitoring systems. The equipment the 
auditors used was not the same as that used for station calibration but 
rather was an independent set. Standard methods, as described in 

Section 3.0, were used to reference all audit standards to NBS SRM. 

4,1 Summary of Audit Results 

Audit data for each of the gas analyzers consist of a set of four to 
six audit concentrations and the corresponding instrument responses. A 
linear regression was run on each data set to determine the best fit 
straight line equation relating the audit value to the analyzer response. 
The result of this regression analysis consisted of three numbers: the 

slope, the intercept and a correlation coefficient. The slope is the ratio 
of incremental change in analyzer response to an incremental change in 
audit input. A value of 1.00 for the slope indicates that the incremental 
responses agree perfectly with the known incremental values of the audit 
concentrations. Slope values greater than 1.00 indicate that the incre- 
mental response of the analyzer is greater than the incremental change in 
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Table 4-1 


SURFACE MONITORING NETWORK FOR 1979 SEV-UPS STUDY 


AEROMETRIC 


SURFACE STATION 

PARAMETERS MONITORED 

AUDITED 

Naval Air Station 
( Inner Norfolk) 

Ozone 

Oxides of Nitrogen 
Sul fur Dioxide 
THC, CO, CH^ 

Yes 

Naval Comm. Center, 
Nor thwes t 

Ozone 

Oxides of Nitrogen 
Sul fur Di oxide 
THC, CO, CH4 

Yes 

Agricultural Station 
(Virginia Research 
Station) 

Ozone 

THC, CO, ai^ 

Yes 

Chesapeake Airport 

Ozone 

Yes 

Hampton School 
(Virginia Schoo 1 ) 

Ozone 

Yes 

Chesapeake Light 

Ozone 

Yes 

Cher i t on 

Ozone 

Yes 

Wachapr eaque 
(Virginia Institute 
of Marine Science) 

Ozone 

Yes 

NASA/Lang 1 ey 

Ozone 

Oxides of Nitrogen 
Sul fur Dioxide 
THC, CO, CH^ 

Yes 

Wallops Flight Center 

Ozone 

Yes 

Tidewater Community 
Coll ege 

Ozone 

No 

Milford Haven 

Ozone 

No 
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input and is, therefore, likely to produce measured values in excess of the 
true value. Slopes within the interval 1.00 ^ 0.05 are generally regarded 
as indicating excellent analyzer performance while variations up to 1.00 
0.15 are considered satisfactory for typical gas analyzers. 

The calculated intercept provides a quantitative measure of the 
analyzer response to a zero concentration input (assuming that the analyzer 
response is linear over its operating range). The intercept is the result 
of the regression calculation and not simply the response of the instrument 
to zero concentration. Intercepts calculated from audit data should not 
exceed j|^3% of the analyzer's range according to usual audit assessment 
criteria used at RTI, 

The correlation coefficient provides an indication of the goodness of 
fit of the line to the data, and includes non-linearity of the analyzer 
response and scatter across the audit concentrations. A value of 1.00 is 
the optimum value, and values between 0.9995 and 1.00 reflect satisfactory 
performance. Values less than 0.9995 indicate varying degrees of instru- 
ment nonlinearity or excessive variation in the data. 

Separate tables containing the audit results are given for ozone 
(Table 4-2), oxides of nitrogen (Table 4-3), sulfur dioxide (Table 4-4) and 
hydrocarbons - carbon monoxide (Table 4-5). 

4.2 Interpretation of Audit Results 

Audit data from a network of analyzers may be used for the statistical 
estimation of accuracy and precision of a monitoring network if a suffi- 
cient number of data points exist. Generally, these calculations require 
either a number of audit points from a single analyzer over a period of 
time or data from several analyzers which were operated uniformly (cali- 
brated by the same team using the same standards). 

Ozone is the only pollutant which was monitored at each of the 12 
ground stations and on board four aircraft. Ten of the 12 surface stations 
taking part in this study and four aircraft were audited by the RTI field 
audit crew. The ten audited ground stations were operated by personnel 
from three different groups: RTI (two stations), NASA (two stations) and 

Virginia State Air Pollution Control Board (six stations). Since these 
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Table 4-2 


RESULTS OF PERFORlvlANCE AUDITS: 


OZONE 


Surface Site 

Slope 

Interce pt 
(ppb) 

Correlation 
Coe fficient 

Bias at 
5 ppb 

Bias at 
50 ppb 

Naval Air Station 

1.01 

-4 

0.9999 

-4 

-4 

Navy CC, Northwest 

1.01 

-2 

0.9999 

-2 

-2 

Chesapeake Light 

1 .02 

-3 

0.9999 

-3 

-2 

Virginia School 

1.01 

-3 

0.9998 

-3 

-2 

Chesapeake Airport 

1.06 

-1 

0.9999 

-1 

+ 2 

Cher i t on 

1.07 

-2 

0.9999 

-2 

+2 

Wachapre ague 

1.01 

+4 

0.9999 

+4 

+ 4 

Agricultural Station 

1.01 

0 

0.9998 

0 

0 

NASA/Langley 

1.03 

-2 

0.9990 

-2 

0 

Wallops Flight Center 

1.01 

+4 

0.9999 

+ 4 

+4 

Ai r era f t 

Cessna 402, ML 

1.52 

+ 2 

0.9981 

■h5 

+28 

Cessna 402, Dasibi 

0.97 

0 

0.9983 

0 

-2 

C-54, Dasibi 

1.03 

-2 

0.9995 

-2 

0 

LAS, Dasibi 

1.01 

-1 

0.9999 

-1 

0 

RTI, Navajo 

1.05 

+ 3 

■0.9998 

+ 3 

+6 



Table 4-3. RESULTS OF PERFORMANCE AUDITS: OXIDES OF NITROGEN 


Surface Site 


NO 



NO 2 



NOx 


Slope 

Intercept 

Cor re 1 . 
Coeff . 

Slope 

Intercept 

Cor re 1 . 
Coeff. 

Slope 

Intercept 

Correl . 
Coe f f . 



(ppb) 



(ppb) 



(ppb) 


Naval Air Station 

0.48 

-2 

0.9999 

1.08 

+7 

0.9996 

1.09 

-8 

0.9998 

Navy CC, Northwest 

1.06 

+ 1 

0.9995 

1.02 

+7 

0.0960 

1.14 

0 

0.9998 

NASA/Langley 

0.65 

-0.5 

0.9980 

0.52 

+ 2 

0.9975 ■ 

0.51 

-4 

0.9953 

Ai rcra f t 










Cessna 402 

i 

1.01 

+2 

0.9995 

0.95 

-2 

0.9995 

0.99 

+ 1 

0.9999 

RTI Navajo 

1.00 

+ 1 

0.9995 

0.98 

-4 

0.9999 

0.96 

+ 2 

0.9996 










Table 4-4. RESULTS OF PERFORMANCE AUDITS: SULFUR DIOXIDE 


Surface Site 

Slope 

Intercept 

Correlation 



(ppb) 

Coe f f i c i ent 

Naval Air Station 

0.8056 

-8 

0.9995 

Navy CC, Northwest 

0.8030 

-1 

0.9997 

Aircraft 




RTI Navajo 

0.9810 

-5 

0.9974 
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Table 4-5. RESULTS OF PERFORMANCE AUDITS: HYDROCARBONS AND CARBON MONOXIDE 


Total Hydrocarbon (0-10 ppm) 
Surface Site Slope Intercept r* 


Naval Air Station 

1.04 

+0.040 

0.9981 

Navy CC, Northwest 
Virginia Research 

0.96 

-0.026 

0.9998 

Center 

0.89 

+0.078 

0.9999 

NAS A/Langl ey 

0.36 

+1.740 

0.6260 


Methane (0-10 
Slope Intercept 

ppm) 

r 

Carbon 

Slope 

Monoxide (0 
Intercept 

-10 ppm) 
r 

1.00 

0.98 

+0.021 

-0.065 

0.9982 

0.0998 

1.05 

1.05 

+0.111 

+0.103 

0.9999 

0.9991 

0.93 

0.96 

-0.075 

+0.030 

0.9999 

0.9994 

0.97 

-0.319 

0.9997** 


Aircraf t*** 


Pol 1 utant 


Analyzer Cylinder Percent 
Response Concent rat ion D1 f f erence 


NASA C-54 Propane 

Total Hydrocarbon 
Me thane 

Carbon Monoxide 


1.32 1.44 -8.3% 
3.27 3.43 -4.7% 
1.95 1.99 -2.0% 
Not applicable 


* r = correlation coefficient 
** 0-100 ppm 

e concentration audit by the chromatograph operator using 
containing propane and methane in synthetic air (21.5% oxygen, 

ference = (analyzer response-audit concentration) ^ 100 


***Results reported are for a singl 
undiluted contents of a cylinder 
balance nitrogen), where: 

percent dif 


(audit concentration) 



three organizations use different standards, different operators and possi- 
bly different procedures, the audit results can be used to assess compara- 
bility of measurements across organizations. As a measure of compara- 
bility, the mean bias was calculated at a typical measurement value of 
50 ppb for each organization. The results of this tabulation are shown in 
Table 4-6. Also included in this tabulation are similar results for the 
aircraft. The aircraft system data were maintained separately from the 
ground station data since aircraft were generally operated independently of 
the ground stations (although some aircraft systems were traceable to the 
same laboratory standards). 

The net result of the ozone audits is that, with only one exception 
(the NASA Cessna ML ozone unit), all analyzers audited exhibited a response 
which was within 10% of the audit value at concentrations typically greater 
than 50 ppb. NASA QC checks of the Cessna ML ozone instrument, after 
installation on board the aircraft but prior to the RTI audits, indicated a 
calibration change relative to laboratory results. Prior to the audit, the 
RTI audit team was notified of this result and, in fact, audit results con- 
firmed the suspected calibration change (data of Tables 4.2 and 4.6). The 
NASA QC and RTI audit results were used to calculate a new calibration 
constant for this instrument; hence, all Cessna ML ozone data presented are 
correct, requiring no corrections based on the audit results. It is 
interesting to note that the mean error computed from audit values is 
lowest for the largest group of stations and increases as the number of 
stations diminish, possibly indicating that the error is random across 
analyzers and that the sample mean approaches a true mean of zero as the 
number of samples increases. 

Other gaseous species monitored in the SEV-UPS program included oxides 
of nitrogen, sulfur dioxide, hydrocarbons and carbon monoxide. Unlike 
ozone, these pollutants were measured at relatively few stations. NO-NO^ 
monitors were operated at two ground stations (Naval Air Station and Naval 
Communications Center) and in the two ^ situ aircraft. Sulfur dioxide 
monitors were operated at the same ground stations and in the RTI aircraft. 
Four real-time hydrocarbon analyzers were used in the SEV-UPS program: 
Virginia Research Center (adjacent to the Agricultural Research Station), 
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Table 4-6. SUMMARY OF OZONE AUDIT DATA AVERAGED BY 

AGENCY OPERATING STATION 


Average difference 

Operating Agency at 50 ppb level 


SURFACE 

STATIONS 

RT I (2 s t a t ions ) 

SAPCB (6 stations) 

NASA Langley (1 station) 
NASA Wallops (1 station) 

-3 

-0.5 
0 
+ 4 


RTI ( 1 analyzer ) 

+ 5 

AIRBORNE 

NASA Cessna (2 analyzers) 

+ 13* 

STATIONS 

NASA C-54 (1 analyzer) 

-1 


NASA CAS (1 analyzer) 

0 


ic 


Average of two 
averages in thi 
of other values 


anal yz er s : 
s chart, -2 
, +28 ppb. 


one consistent with other 
ppb; and the other in excess 
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Naval Air Station, Naval Communications Center, and on board the C-54 air- 
craft. All of these analyzers (NO^, SO 2 and Hydrocarbons) were audited 
during the same time period as the ozone instruments. However, since there 
were few instruments of each type in operation, and no more than two in 
operation by any one operating organization, it would be meaningless to 
calculate averages of noted bias as was done for ozone. Rather, it is more 
meaningful to examine the individual bias observed, comment on reasons for 
noted discrepancies and describe any action taken to correct discrepancies. 

Oxides of nitrogen analyzers in the aircraft generally exhibited 
excellent agreement with audit systems (within 5% as shown by the slope). 
Analyzers at the ground stations did not exhibit the same level of 

performance. The analyzers at the Naval Communications Center showed 
excellent agreement with the audit system on NO; however, performance was 
marginal on the NO^^ channel as indicated by the slope of 1.14. Intercept 
values for NO and NO^ channels on this analyzer were acceptable. This 
high slope value, if not corrected, would not cause errors of greater than 
1 ppb, since measured ambient values did not exceed 10 ppb except for one 
hourly period during the month. The Naval Air Station NO^ analyzer 

exhibited poorer performance during the audit with a slope of 0.48, indica- 
ting the calibration of the analyzer was off by a factor of nearly two. 
The analyzer was recalibrated after the audit, thus eliminating the 
problem. The NO^ analyzer at the NASA/Langley Research Center was found 
to have a clogged orifice which prevented proper operation of the analyzer. 
The orifice was cleaned after the audit and the analyzer recalibrated. 

Three sulfur dioxide instruments were audited. The sulfur analyzers 
at the two ground stations operated by RTI exhibited the same error in 
slope. This error resulted from an incorrectly verified calibration 

cylinder. The cylinder concentration was rechecked and all reported data 
were processed using the correct result. The RTI aircraft SO 2 analyzer 
was operated without the electronic linearizer to facilitate altitude 

correction of the data. Linearization was accomplished off-line during 
data processing. The coefficients for the linearizing equation were 
determined as a part of the routine calibration procedure which, during 
this study, was conducted every two or three days. The correction 
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coefficient computed from the audit data for this analyzer was 0.98, which 
differed from the desired value of 1.00 due to inaccuracies in the lineari- 
zation compensation at the time of the audit. The error attributable to 
this problem is -10 ppb at concentrations of 40 to 70 ppb with the magni- 
tude of the error decreasing to zero as the measured concentration 
approached zero. After the completion of the audit, the instrument was 
recalibrated and new coefficients calculated to rectify this problem before 
any data were collected. 

The results of the hydrocarbon analyzer audits showed the analyzers at 
the two ground stations operated by RTI to be within criteria established 
for "excellent". The analyzer at the Virginia Research Station was satis- 
factory with a span shift of approximately 10% for the total hydrocarbon 
measurement and 8% for methane. The intercepts for all channels of the 
above analyzers were all within acceptable limits for air monitors. The 
only analyzer which appeared unsatisfactory according to the audit was the 
analyzer at the NASA station which was in need of repair at the time of the 
audit. 


4.3 Conclusions 

The ozone audit data indicate that, generally, the ozone analyzers 
were operated and calibrated v/ithin satisfactory limits. Only in one 
specific case did the audit results indicate a problem, and this analyzer 
was recalibrated so the unsatisfactory audit results do not reflect the 
accuracy of the reported data. 

Average values of ozone concentration error at the 50 ppb level were 
computed across each organization operating ground stations. No signifi- 
cant bias was found for any operating organization. Audit data for other 
analyzers generally showed excellent analyzer performance or indicated a 
problem which was later verified. The problem was either corrected before 
data were reported or the data were invalidated. Consequently, there are 
no known instances where audit data indicated a significant error in the 
data base. 
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5.0 COMPARISON OF COLLOCATED MEASUREMENTS FROM 
DIFFERENT MEASUREMENT SYSTEMS 


Audits of analyzers such as those described in Section 4.0 verify the 
calibration of the analyzer and offer assurance that the analyzer is 
functioning properly. However, the actual operating conditions of the 
analyzer usually differ from conditions of the audit. For example, in the 
case of airborne systems, the instrument is audited on the ground and 
powered by ground line voltage. The test gas concentration is supplied 
from dry, scrubbed compressed air and the output is measured by reading the 
output voltage with a voltmeter. During actual operation, the instrument 
will be operated in a vibrating aircraft, powered by inverters, sampling 
air with varying humidity. In addition, the ambient air will be delivered 
to the instrument through the aircraft air intake system and data are 
recorded using some form of data acquisition system, neither of which were 
tested by the audit. 

One technique of verifying the operation of the total system is to 
operate it in a normal manner in a known environment. This can be 
accomplished by determining levels within the ambient environment by using 
a collocated independent technique which has been proven reliable. 
However, there are presently no available systems for hi situ measurements 
over the altitude range of 0 to 3000 m (0 to 10,000 ft) which are better 
proven than aircraft systems. Therefore, no standard exists by which an 
aircraft may be easily compared. As an alternative, two or more aircraft 
(or one aircraft and another type of platform) may be compared in order to 
determine the degree of consistency of their measurements. This type of 
analysis must be performed with care, however, since under certain condi- 
tions, differences may be expected between measurements made by different 
platforms at nearly the same location and time. This is true particularly 
in the case where aircraft readings may be compared against measurements 
taken from ground stations. 

The SEV-UPS data base is composed of data collected by four different 
aircraft, two tethered balloon systems and twelve ground stations. To test 
this data base for consistency, periods were identified when two or more 
monitoring systems were simultaneously operating in close proximity. Data 
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values were obtained, plotted and processed using statistical software 
packages. The procedures which were used are presented in the next section 
and the results are presented in the following section. 

5.1 Procedure 

The data comparison effort began with the identification of instances 
when two or more measurement platforms were operating in the same area at 
approximately the same time. Most of the identified cases arose from one 
of three instances: 

1. Spirals performed by two aircraft at the same location 

within 30 minutes. 

2. A traverse by two aircraft along the same ground track at 

the same altitude at approximately the same time 
period (within one hour). 

3. A low pass of an aircraft near a ground station. 

To aid in the identification of these comparison cases, bar charts were 
constructed showing times of available data for each of the mobile plat- 
forms. At this point, the ground stations were assumed to provide con- 
tinuous data. Next, the flight plans of the experiments performed were 
examined to determine the times comparison cases were to have taken place. 
Flight descriptions from each of the participants were then examined to 
determine if the flight pattern flown was consistent with the plan. 
Finally, the times of data availability were examined to determine if data 
were available from both monitoring systems at the time in question. Since 
point values were not available at ground stations, hourly averages were 
used. When hourly averaged values were compared with point values falling 
within 10 minutes on either side of the hour, an averaged value for the two 
nearest hourly periods was used. 

Since the primary purpose of this effort was to identify any bias 
which might exist between pairs of monitoring platforms, averaging was used 
to reduce the quantity of data to be processed. Data from corresponding 
intervals from two or more systems were averaged to a single value, thus 
avoiding the problems associated with comparison of data acquired at 
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II 


different sampling rates. No attempt was made to examine the relative 
variances of each data set. Vertical profile data were averaged over 200 -m 
altitude ranges to prevent the loss of bias information which was related 
either to altitude or to the level of the measurement (which often varied 
with altitude). 

The resulting data set was checked and edited until all identifiable 
errors were removed on the RTI PDF 11/60 computer. The edited data set was 
V then transferred via phone lines to the Triangle Universities Computation 
Center (TUCC) where it could be processed using any of the statistical 
software packages which are maintained at that facility. 

The data were first processed to determine if any of the airborne 
systems exhibited an altitude-related bias. Plots were generated of the 
difference between the two platforms versus altitude. All data for each 
pair of monitoring platforms were plotted on a single plot to avoid the 
drawing of erroneous general conclusions from a single case hypothesis. 
Data plots exhibiting a visual trend were subjected to a regression 
analysis. 

Next, the altitude was removed as an analysis variable and all pairs 
of comparison data for each combination of two monitoring systems were 
plotted on a scatter graph representing concentration from System A versus 
concentration from System B. Both linear regression analyses and simple 
calculation of average differences were used to characterize observed 
differences in measured values. 

5.2 Data Comparison Results 

The preliminary review of data summaries revealed that on seven days 
during the 1979 summer SEV-UPS program, two or more of the air monitoring 
systems were operated in close proximity in such a manner that good agree- 
ment was expected. On these seven days, a total of 21 cases for comparison 
data were identified. Each case consisted of at least one pair of average 
data values for comparisons at one altitude and more points for comparisons 
which encompassed a range of altitudes. Table 5-1 presents the list of 
comparison cases identified during this preliminary review. In the actual 
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Table 5-1 


AVAILABLE SEV-UPS COMPARISON DATA 


DATE 

(COhlPARISON CXDMPARED SYSTEMS TIME OF COMPARISON CIRCUMSTANCES DATA 

NUMBER) 


August 

(1) 

15 

NASA Cessna 
NASA C-54 

0844-0853 

0834-0900 

Spiral at Lake 
Drummond (Swamp 
Character i zat ion ) 

7 average points at 
7 altitude ranges 

(2) 


NASA Cessna 
NASA C-54 

1020-1048 

1018-1044 

Comparison Flight 

10 average points at 
8 altitude ranges 

( 3) 


NASA Cessna 
Ground Stations: 

Chesapeake Light 1444-1454 
Naval Comm. Center 1516-1526 
Naval Air Station 1538-1551 
Langley Research Center 1558-1608 

Cessna Ground 
Comparison Flight 

No data 

One data point 
One data point 
No data 

Augus t 
(4) 

20 

RT I Nava j o 
NASA Cessna 

1115-1144 

1112-1145 

RTl /NASA Cessna 
Comparison Flight 

10 average points at 
8 altitude ranges 

Augus t 
(5) 

24 

NASA C-54 
RT I Navajo 

w0650,«0805 

0719-0729 

Spiral at Naval 
Comm. Center, Urban 
Plume Experiment 

8 average points at 
8 altitude ranges 

(6) 


RTl Navajo 
NASA C-54 

1233-1246 

«1311 

Spiral in Leg EF 
Urban Plume 
Experiment 

8 average points at 
8 al t i tude ranges 

( 7) 


RTl Navajo 
Ground Stations: 

Chesapeake Airport 

1115 

Low passes conducted 
during Urban Plume 
Exper imen t 

1 average point at 


ground level 
No data 


Chesapeake Light 


1304 



Table 5-1. Continued 


DATE 

(COMPARISON COMPARED SYSTEMS TIME OF COMPARISON CIRCUMSTANCES DATA 

NUMBER) 


August 25 

RTI Navajo 

0602-0615,0730-0743, 

Spirals at Naval 

24 average 

points at 

(8) 


and 1059-1110 

Comm. Center (Urban 

8 altitude 

ranges 


NASA C-54 «0650, «0805, wll50 

Plume Experiment) 



(9) 

RT I Nava j o 

1215-1230 

Spiral in Leg EF 

8 average 

points at 


NASA C-54 

«1247 

(Urban Plume 

8 altitude 

ranges 




Experiment ) 



(10) 

RTI Navajo w0602, ss0731, wl059 

Low Passes Conducted 

3 values at ground 


Chesapeake Airport 


during Urban Plume 

leve 1 





Study 



August 29 

RTI Navajo 


Low Pass Comparison 




Ground Stations: 


FI ight 



(11) 

Langley Research Center 1018-1033 


No data 



Naval Air Station 

1040-1053 


1 value at 

ground level 


Chesapeake Airport 

1107-1114 


1 value at 

ground level 


Naval Comm. Center 

1119-1125 


1 value at 

ground level 


Chesapeake Light 

1155-1200 


1 value at 

ground level 

( 12) 

RTI Navajo 

1119-1134 

Spiral near N.C.C. 

2 average 

points at 


Balloon at N.C.C. 

1127-1141 

during Low Pass 

2 altitude 

ranges 




Comparison Flight 



(13) 

NASA Cessna 

1045-1107 

Balloon Comparison 

2 average 

points at 


Bal loon at N.C.C. 

1147-1158 

FI ight 

2 altitude 

! ranges 

(14) 

NASA Cessna 

1156-1217 

Spiral during 

7 average 

points at 


Balloon at Wallops 

1244-1321 

Balloon Comparison 

7 altitude 

( ranges 




Flight 



(15) 

NASA Cessna 


Low Passes from 




Ground Stations: 


Balloon Comparison 




Naval Comm. Center 

1035-1045 

Flight 

1 average 

point at 





ground level 


Chesapeake Light 

1121-1133 


1 average 

point at 


ground level 



Table 5-1 


Concluded 


DATE 


(COMPARISON 

COMPARED SYSTEMS 

TIME OF COMPARISON 

CIRCUMSTANCES 


DATA 

NUfvlBER) 






Augus t 

30 

RTI Navajo 

0729-0743 

Urban Plume 

8 

average points at 

(16) 




(Option 1) 

8 

altitude ranges 



Balloon at N.C.C. 

0747-0815 

Spiral in Leg AB i 

(3 

points for balloon) 



NASA C-54 

a0756 

(Naval Comm. Center) 



(17) 


RT I Nava j o 

1116-1128 

Urban Plume 

3 

average points at 





(Option 1) 

3 

a 1 1 i t ude ranges 



Balloon at N.C.C. 

1107-1128 

Spiral in Leg AB 







(Naval Comm. Center) 



(18) 


RT I Nava j o 
Chesapeake Airport 

0600, 0730, 1116 

Low passes during 

3 

averages at ground 





Urban Plume 
Ex per iment 


1 evel 

Augus t 

31 

RT I Nava j o 

0749-0758 

Spiral at Point F 

8 

average points at 

(19) 


NASA Cessna 

0800-0810 

Photochemical Box 

8 

altitude ranges 





Experiment (Option 7] 

) 


(20) 


RT I Nava j o 

1106-1108 

Leg DE of 

1 

average point at 



NASA Cessna 

1042-1100 

Photochemical Box 
Experiment (Option 5; 

1 

) 

altitude range 

(21) 


RTI Navajo 

1410-1414 

Leg DE of 

1 

average point at 



NASA Cessna 

1412-1415 

Photochemical Box 

1 

al t i tude range 


Experiment (Option 5) 



tabulation of data, it was noted that all data within each comparison case 
was not always available due to equipment failure. An additional compari- 
son, derived from the identified comparison data, was made between the 
balloon and ground at the Naval Communications Center. 

Certain of the identifiable comparison data originated from flights, 
or portions of flights, which were incorporated into the SEV-UPS plans for 
the express purpose of providing comparison data between aircraft and the 
ground stations or between different aircraft. These data were examined 
first before being incorporated into the data base with all other compari- 
son data. 

Data from all comparisons, including low passes and comparison 
flights, were grouped into a single data base and processed by the computer 
using the Statistical Analysis System (SAS) for generating plots of the 
data, fitting lines to comparison pairs of data points and performing 
statistical calculations of the data. The data were first tested for bias 
which varied as a function of altitude. Then the data from each system 
were compared against all other systems having data taken at the same time 
and place. The results of this comparison are presented in Section 5.4. 

5.3 Low Pass Comparisons 

Low pass comparisons provide opportunities for airborne air sampling 
systems to acquire data which may be compared to data from ground stations 
which use established, proven sampling techniques. However, the majority 
of the low pass ozone data taken during the 1979 program seem to verify 
that comparable data may be obtained by this technique only under certain 
circumstances when mixing is sufficient to insure the aircraft and ground 
station are sampling identical air parcels. In the number of cases 
available for this comparison, one v^^ould expect to find some cases where 
good agreement is obtained. However, the low pass data from both the RTI 
and the NASA Cessna aircraft (Tables 5-2 and 5-3) indicate that there were 
no cases which showed good agreement. This poor agreement could be attri- 
buted to ozone scavenging by ground level vegetation, buildings and other 
ground coverings. This finding does not appear to indicate measurement 
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Table 5-2. LOW PASS** OZONE COMPARISON SUMMARY - RTI AIRCRAFT 


Station 

Date 

Time 

Aircraft Data 

Station Data 

Ccrrments 




(ppb) 

(ppb) 


Chesapeake 

8/25 

1247 

41* 


36 ppb was Icwest 

Light 



39* 


value in spiral 




36 

23 



8/29 

1155 

36 

17 

Ozone values did not 



1158 

35 


vary significantly 



1201 

36 


below 200 m 


8/30 

1305 

102 

98 

Flyover (not a low 






pass) 

altitude = 400 m 

Chesapeake 

8/24 

1115 

49 

25 

Values at 200 m were 

Airport 





55 ppb - slightly 
higher than ground 


8/25 

0602 

11 

0 

Values near ground 



0731 

11 

0 

were very low in com- 
parison to upper air 
50 ppb at 1200 m 


8/25 

1059 

36 

15 (10:00) 

Values at ground 





25 (11:00) 

higher than earlier 
but vertical 
gradient still exists 


8/29 

1107 

42 

20 

Values consistent to 



nil 

42 


above 300 m 


8/30 

0601 

7 

0 

Strong vertical 



0729 

7 

0 

gradient - O3 
concentration - 60 ppb 
at 300 m 


8/30 

1116 

61 

35 

Ozone relatively well 






mixed - 10 ppb 
difference ground to 
400 m 

Naval 

8/29 

1119 

41 

24 

Ozone relatively well 

Conmunica- 


1121 

41 


mixed - 10 ppb 

tions 


1124 

41 


difference ground 

Center 





to 400 m 

Naval 

8/29 

1040 

46 


Values at 300 m were 

Air 



40 


10 ppb higher 

Station 



38 

14 




1047 

39 





1053 

42 




* Measured values taken irrmediately prior to low pass. 


** All values from aircraft were acquired within approximately 10 m of the ground 
station sarrple intake elevation except the flyover at Chesapeake Light. 
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IPARISON SUMMARY - NASA CESSNA 


Aircraft Data Station Data 




problems since the degree of difference is about the same for both aircraft 
and all ground stations, whereas a measurement-related difference of that 
magnitude would not be as consistent over all measurement systems and would 
have been detected by the audits. 

One interesting note concerning the ozone comparison data is that the 
best ground station-aircraft agreement occurred during a flyover at the 
Chesapeake Light which was initially identified as a low pass. However the 
altitude of the aircraft was approximately 400 m (1200 ft) above the 
station. Data along the flight path at the same altitude showed relatively 
little variation in ozone values. Spiral data, taken over the Chesapeake 
Bay in an attempt to verify the mixing, showed that area to be influenced 
by local plumes containing and sulfur dioxide which caused signifi- 

cant variations in ozone concentrations. However, those plumes were not in 
evidence near the vicinity of the Chesapeake Light at the altitude flown. 

Low pass comparison data for the measured variables other than ozone 
are available in smaller quantities since only two operational ground sta- 
tions monitored oxides of nitrogen, temperature and dew point. This com- 
parison data, summarized in Table 5-4, generally illustrates better agree- 
ment between aircraft and ground measurements than the ozone data. 
However, one potential problem area is identified here: the NO and NO^ 

measurements made on the NASA Cessna appear to be biased approximately 
+15 ppb. This fact is also substantiated in interaircraft comparisons. 

5.4 Aircraft Comparison Flights 

The best source of data for comparing the airborne monitoring systems 
is the aircraft comparison flight pattern (Figure 5-1) where two aircraft 
fly in formation and acquire data during straight and level flight seg- 
ments. A spiral in formation was also included in this test to compare 
instrumentation operation in a descending spiral pattern frequently encoun- 
tered during the SEV-UPS field program. During the comparison flight both 
aircraft measurement systems were operated in the manner for which they 
were designed : stable flight with no frequent abrupt changes in levels of 

the species they were measuring. Since measurement systems are similar. 
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Table 5-4. LOW PASS COMPARISON SUMMARY FOR MEASUREMENTS OTHER THAN OZONE 


Temperature 

Dew Point 

(°C) 

(°C) 

Aircraft Station 

Aircraft Station 

24.1 24.0 

10.7 12.8 


Station 


Date - Time 


NASA CESSNA 


Naval Corrm. Center 
8/15 1516-1526 

Naval Air Station 
8/15 1538-1551 

Naval Carm. Center 
8/29 1035-1045 


ID 


(PPb) 


Sc 

(ppb) 


RTI NAVAJO 
Naval Air Station 
Naval Corrm. Center 


28.6 28.6 




















one would expect to obtain good agreement in the comparison data. 

Table 5-5 cites the comparison data for ozone, the only common 
measurement, taken on both aircraft during a comparison flight conducted on 
August 15, 1979. Ozone data compares reasonably well between both air- 
craft, showing only an average difference of 6 ppb if the uppermost point 
is excluded from the comparison. The average bias becomes 7.2 ppb when 
this point is included. 

Table 5-6 cites the comparison data acquired during the only other 
instrumented aircraft comparison flight performed during the SEV-UPS study. 
This flight was made by the RTI Navajo and the NASA Cessna on 
August 20, 1979. This flight compared two aircraft with a similar comple- 
ment of instrumentation. The ozone measurements indicate a negative bias 
for the RTI aircraft relative to the NASA Cessna. The NO and NO^ data 
substantiate the 15 ppb positive bias exhibited by the NASA Cessna low pass 
data discussed earlier. The significant difference shown in 
data might be due to nephelometer sample preheaters used on the NASA Cessna 
{and not on the RTI aircraft). These heaters vaporize precipitating 
moisture so it is not detected by the analyzer as light scattering parti- 
culate matter. For this reason the RTI analyzer is more sensitive to water 
vapor droplets. On this comparison flight this difference is further 

exaggerated because the RTI aircraft flew closer to clouds than did the 
NASA Cessna. 


5.5 Composite Data Comparison 

The collected comparison data included low pass data, aircraft com- 
parison data, and data from periods when two or more monitoring systems 
were operating near the same location when performing routine data collec- 
tion. Data from all sources were consolidated into a single data set con- 
taining data organized so that the originating monitoring system, the 
time/location and the altitude of each data point could be identified. 

Initially, plots of difference values (difference between values 
reported by two different monitoring systems) were plotted versus altitude 
to test the data for bias which were a function of altitude. Since only 


37 



Table 5-5. NASA CESSNA 


C-54 FLIGHT COMPARISON 


OZCM OSi^CTNIRATION 

NASA Cessna NASA C-54 
(ppb) (ppb) 


Difference 

Cessna-C-54 

(ppb) 


1500 m segnent 

54 

61 

- 7 

Spi ral 




1500-1400 m 

47 

63 

-16 

1400-1200 m 

51 

54 

- 3 

1200-1000 m 

48 

53 

- 5 

1000- 800 m 

47 

52 

- 5 

800- 600 m 

46 

54 

- 8 

600- 400 m 

47 

54 

- 7 

400- 200 m 

46 

52 

- 6 

Below 200 m 

46 

51 

- 5 

600 m segnent 

46 

54 

- 8 


Average A = - 7.0 



NAVAJO FLIGHT COMPARISON 


Parameter 

Terrperature Dew Point 

(10-4/m) (°C) (OC) 

DIFF. ; NASA RTI DIFF. I^\SA RTI DIFF. I^RSA RTI DIFF. 


.6 


no 


1.7 

2.0 

- 0.3 

2.1 

1.6 

0.5 

1.4 

3.3 

- 1.9 

1.9 

4.2 

- 2.3 

1.6 

3.7 

- 2.1 

2.3 

6.4 

- 3.9 

2.8 

5.6 

- 2.8 

3.2 

4.4 

- 1.2 



- 1.41 


. 5.3 15.2 0.1 

. 6.1 16.4 - 0.3 




values frcm NASA values 

















airborne platforms (including the tethered balloon) were involved, only 
eight (8) comparisons involving 4 monitoring systems were made which con- 
sisted of points at more than one altitude. There was only one identi- 
fiable case where a bias was noted which varied repeatedly with altitude. 
The RTI Navajo/C-54 ozone comparison data show a slight increasing trend in 
difference values with an increase in altitude. The change amounted to 
11.75 ppb per 1 km altitude with a correlation of about 0.5. The plot of 
the Navajo/C-54 data is shown in Figure 5-2, The remaining altitude com- 
parisons presented no noticeable altitude dependence, and therefore, only a 
single representative example plot shows the difference between ozone 
response for the RTI Navajo and NASA Cessna versus altitude (Figure 5-3). 
A regression analysis indicated a correlation of only 0.02 once the outlier 
of 17 ppb at 300 m was removed. 

Since differences between measurements made by different systems were 
generally not a function of altitude, the data from all altitudes were 
combined and plotted on scatter plots. The data compared in each plot and 
the figures containing the plot are as follows: 

Ozone : 

NASA Cessna Versus RTI Navajo 
NASA Cessna Versus C-54 
RTI Navajo Versus C-54 

Balloon at Wallops Island Versus NASA Cessna 
Balloon at NCC Versus RTI Navajo and NASA 
Cessna 

All Ground Stations Versus RTI Navajo 
All Ground Stations Versus NASA Cessna 
Balloon at NCC Versus Ground Station at NCC 

NO: 

NASA Cessna Versus RTI Navajo and Ground 
Stations at NCC and NAS 
NAS and NCC Versus RTI Navajo 

NOx: 

NASA Cessna Versus RTI Navajo and Ground 
Stations at NCC and NAS 
Ground Stations at NAS and NCC Versus RTI 
Navajo 


Figure 5-4 
Figure 5-5 
Figure 5-6 
Figure 5-7 

Figure 5-8 
Figure 5-9 
Figure 5-10 
Figure 5-11 


Figure 5-12 
Figure 5-13 


Figure 5-14 
Figure 5-15 
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Temperature : 

NASA Cessna Versus RTI Navajo and Ground 
Stations and NCC and NAS Figure 5-16 

Balloon and NCC Versus RTI Navajo, NASA Cessna 
and Ground Station at NCC Figure 5-17 

Dew Point: 

NASA Cessna Versus RTI Navajo and Ground 
Station at NCC Figure 5-18 

Balloon at NCC Versus RTI Navajo and 
NASA Cessna Figure 5-19 

®scat ' 

NASA Cessna Versus RTI Navajo Figure 5-20 

In most cases, particularly those involving gas analyzers, the 
comparison data consist of a number of ambient data points which cover only 
a small portion of the instrumentation measurement range. Therefore, a 
linear estimator of the relationship between measurements made by the two 
monitoring systems would probably be accurate only in the region near the 
ambient data points. For ozone, the results of a linear regression on data 
values from each platform (where represents values from one system and 
y^^ represents values from another) would be accurate only in the region 
about the ambient points typically 50 to 80 ppb, A more accurate 
assessment of measurement bias can be computed from the average difference 
of measured values taken at the same point in time and location. 
Consequently, this procedure was used in place of regression analysis. 

There are certain discrepancies which were noticeable in the data. 
Table 5-7 summarizes the average differences in ozone for compared systems 
where two or more comparison data points existed. In all cases, reasonable 
agreement existed (within 20%), except in cases involving a ground station. 
In these cases, good agreement was not expected except in cases where the 
atmosphere was exceptionally well-mixed. Previous studies have shown this 
to occur only under certain circumstances which are not well-defined but 
include such conditions as high wind speed, the lack of low-lying inver- 
sions and the absence of local source plumes. Except in the case of air- 
craft-to-aircraft comparisons, there were usually insufficient points to 
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OZONE RESPONSE DIFFERENCE RTI NAVAJO - NASA CESSNA (ppb) 2 OZONE RESPONSE DIFFERENCE RTI > C54 (ppb) 


Regression Line 


-TOf- O 
O 


Results of linear regression 

aOzone (ppb) = 11.76 x altitude (meters) - 0.76 


ALTITUDE IN METERS 

Plot of Difference Between Ozone Response for 
RTI Navajo and NASA C-54 Versus Altitude 


ALTITUDE IN METERS 

Plot of Difference Between Ozone Response for 
RTI Navajo and NASA Cessna Versus Altitude 


Figu re 5-3 . 













GROUND STATION NASA CESSNA 


Figure 





Comparison of NO Data: NASA Cessna Versus 

RTI Navajo and Ground Stations at NCC and NAS 



-13. Comparison of NO Data: Ground Stations at NAS 

and NCC Versus RTI Navajo 







Figure 5-15. 


Comparison of NO^ Data: Ground Stations at NAS 

and NCC Versus RTI Navajo 





Figure 5-16. Comparison of Temperature Data: NASA Cessna 

Versus RTI Navajo and Ground Stations 
at NCC and NAS 



AIRCRAFT OR GROUND STATION 


Figure 5-17. Comparison of Temperature Data: Balloon at NCC 

Versus RTI Navajo, NASA Cessna and Ground 
Station at NCC 
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Figure 5-20. Comparison of Data: NASA Cessna 

Versus RTI Navajo 

determine the general relationship between measured parameters. In the 
NASA Cessna to RTI Navajo comparison, all points have a consistent rela- 
tionship with one exception: the data point at (92 ppb, 76 ppb) arises 

from a common leg flown as a part of the urban box experiment by the two 
aircraft. There is no known reason why the deviation for this point should 
be any greater (or less) than other differences in the same data set. 

An interesting observation concerning ozone data for the three air- 
borne systems is that all observations are generally consistent; that is, 
measured values from the NASA Cessna were consistently above those of the 
RTI Navajo, and those of the C-54 were consistently above those of the NASA 
Cessna. However, the measured values from the C-54 were consistently below 
those from the RTI Navajo, (i.e., for ozone, the NASA Cessna read higher 
than the Navajo which read higher than the C-54 which read higher than the 
NASA Cessna). Although this situation seems contradictory and impossible. 
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Table 5-7. COMPOSITE OZONE COMPARISON DATA 


r ' -- - ;r; — •; . — - 



Means of Reported Values 

Difference 






(ppb) 

Between System #1 

Percent 

Nurrber of 

System #1 

System #2 

System #1 

System #2 

and System #2 

Difference* 

Points 

Bal loon 

NASA Cessna 

48.5 

42.0 

6.5 

14.4 

2 

Bal loon 

Naval Cbrrmuni cat ions 







Center 

46.5 

25.5 

21.0 

58.3 

6 

Balloon 

Balloon 

RTI Navajo 

55.8 

53.7 

2.1 

3.8 

5 

(Wallops) 

Chesapeake 

NASA Cessna 

44.3 

41.9 

2.4 

5.6 

7 

Airport 

RTI Navajo 

14.4 

29.1 

- 6.7 

67.5 

8 

NASA Cessna 

Naval Gbrrrmni cat ions 







Center 

64.0 

47.5 

16.5 

29.6 

2 

NASA Cessna 

RTI Navajo 

87.0 

74.6 

12.4 

15.3 

20 

NASA Cessna 

NASA C-54 

49.4 

57.0 

- 7.6 

-14.3 

17 

RTI 

NASA C-54 

53.8 

45.1 

8.7 

17.6 

56 


* Percent difference corrputed by: 


% difference = 2 ( System #1 


System #2 ) /( System #1 + System #2 ) 




I 


it cannot be resolved with available data since all comparison measurements 
between aircraft were only possible by grouping two at a time for cases 
where time and location were approximately the same. Never was there an 
instance where all three aircraft were flown at approximately the same 
location and time. A possible explanation for this contradiction might lie 
in the fact that NASA C-54 comparisons were only made during the early 
stages of the program, while other comparisons were made over the duration 
of the program.. Changes or shifts in calibration over the duration of the 
program for either the RTI or C-54 analyzers would be reflected in one 
comparison set and not in the other. 

The only noted consistent discrepancy appeared in the NO and NO^ 
data for the NASA Cessna aircraft. Mean differences between the NASA and 
the RTI aircraft show the NASA NO values to be biased + 13.2 ppb and the 
NO^ values bias + 14.6 ppb. Low pass comparisons between the NASA Cessna 
and ground stations support this observation (see Figure 5-12 and 5-13). 
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6.0 CONCLUSIONS 


There is no doubt as to the importance of field measurement programs 
to improve our understanding of the interaction of trace pollutants in the 
atmosphere. It is just as important, however, to understand the limita- 
tions of both the instrumentation and the available platforms on which the 
instruments are mounted. Quality assurance programs provide a mechanism by 
which measurements from various platforms can be compared to one another. 
The common reference point from which measurements can be compared is par- 
ticularly important when measurement platforms of such a different nature 
as fixed ground stations and instrumented aircraft are involved in the same 
program. Not only are these two types of systems operated under quite 
different dynamic conditions but they are also operated with different time 
constants and with different time averaging of the output signal. Quality 
assurance reports insure that there is consistency between the performance 
of the instruments when challenged by a known concentration in a controlled 
setting. 

The instrument audits performed as part of the quality assurance 
effort of the 1979 SEV-UPS program identified certain instruments that were 
producing unacceptable responses due to such problems as calibration offset 
or clogged inlet plumbing. These problems, once identified, were easily 
corrected early in the program illustrating a primary advantage of 
including a quality assurance effort in any field measurement program. 

The search for measurement bias among the instruments on the different 
platforms revealed another primary advantage of the quality assurance 
effort. Any bias discovered between the measurements obtained from dif- 
ferent platforms during the field measurement exercise can be assumed to 
result from sources other than those related to the operation of the 
instrument itself. The other possible sources of measurement bias are many 
and include principally effects due to inlet manifold systems, data 
recording systems, power supplies, and atmospheric inhomogeneities in the 
distribution of pollutant species. 

The specific cases of measurement bias between the various platforms 
have been identified and discussed previously. At this time it would be 
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beneficial to discuss those cases of measurement bias that were identified 
as they relate to implications concerning data analysis efforts. 

An important comparison in the SEV-UPS data is the low pass informa- 
tion where two of the aircraft collected data in close proximity to several 
of the ground stations. Although many of the low passes were conducted at 
times that are normally associated with good mixing conditions, a con- 
sistent offset was measured between the ozone concentrations determined by 
the ground station and aircraft platform.. In all cases the difference 
between the measurements was in the range of approximately 10 to 30 ppb. 
This was true for both aircraft and at all surface stations. The aircraft 
measurements which were instantaneous values were always higher than the 
hourly averaged surface measurements. A definite cause of this measurement 
offset cannot be determined by merely inspecting these data. However, it 
most certainly results from a combination of the effects of ozone destruc- 
tion near the surface and time averaging of the signals. 

The consistency of this offset at each surface station and for both 
aircraft, along with the audit results, serve as evidence that the nature 
of the measurement bias was not solely instrumental. During several of the 
low pass cases the aircraft was flown near the surface station more than 
once within a few minutes. In all of these cases the replicate measure- 
ments from the aircraft showed a high degree of precision, which is another 
indication that the instrument was operating properly. This offset was 
observed at the upwind rural areas as well as at the downwind station in 
the surface network. Therefore both the background ozone concentration and 
the photochemically generated ozone concentration can be underestimated by 
the surface measurements. 

Data analysis implications concerning the lateral distribution of 
ozone concentrations can be inferred in a relative sense, however, from the 
surface station network since the offset was so consistent across all low 
pass locations. 

Since the data collected aboard the airborne platforms is a signifi- 
cant amount of the information generated by a program such as SEV-UPS, 
specific aircraft comparison flights were planned so that simultaneous data 
from the various aircraft would be available. Two specific comparison 
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flights were completed^ one involving the NASA Cessna and NASA C-54, and 
the other involving the NASA Cessna and RTI Navajo. 

Measurements were made for comparison purposes in both cases during 
spirals to determine if there was any altitude dependent bias between the 
platforms. All offsets discovered were consistent for all altitudes, 
eliminating altitude dependence as a source of measurement bias. If 
flights which were not specifically designed to be comparison flights are 
included, ozone comparison data relating each aircraft to each other are 
available. In general, the offsets are small for ozone concentrations and 
are consistent for measurements taken on individual days. Over all days on 
which measurements were made, however, there is no consistent relationship 
among all three aircraft. Therefore it must be assumed that there is some 
variance or uncertainty in ozone measurements obtained aboard aircraft 
platforms. Much more data would be required to quantify that variance but 
the data available indicate that the day-to-day variance of ozone measure- 
ments aboard any one aircraft is on the order of 10 ppb. This fact is also 
useful in explaining some of the variation in the bias observed between 
surface and aircraft ozone measurements. 

The only other significant bias that was observed between the aircraft 
platforms was for NO and NO^^ data collected on the NASA Cessna and RTI 
Navajo. This particular offset, which was seen throughout the data set, 
indicated that the NASA Cessna measured NO and NO^ values consistently 
higher than the RTI Navajo. (Average difference was 13 ppb for NO and 
15 ppb for NO^.) This is particularly important considering the fact 
that the RTI measurements were nearly always less than 5 ppb, the minimum 
detectable limit of the instrument used. A similar offset was also 
observed in the Cessna-surface station low pass comparisons. The fact that 
this offset was so consistent, and was not discovered in the quality 
assurance audit, indicated that the cause was not due to instrument mal- 
function. NASA later determined that the source of error was located in 
the data acquisition system. 

In general, the quality assurance program of the 1979 SEV-UPS improved 
the data recovery by pointing out measurement problems associated with the 
operation of selected instruments early in the program. It also allowed 
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some potential causes of discrepancies between aircraft and surface ozone 
measurements to be eliminated. In addition, the quality assurance program 
also confirmed that the error in the Cessna NO data originated from the 
aircraft data acquisition system. It was also possible to estimate, in a 
very crude way, the day-to-day variance expected among aircraft ozone 
measurements. From a data analysis standpoint it showed that surface ozone 
measurements can be used to represent the relative lateral distribution of 
ozone but will nearly always underestimate the ozone concentration 
immediately above the surface. 
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